Skip to content

SNOW-2364943: support export external telemetry to event table#3813

Merged
sfc-gh-yuwang merged 37 commits into
mainfrom
SNOW-2364943
Oct 24, 2025
Merged

SNOW-2364943: support export external telemetry to event table#3813
sfc-gh-yuwang merged 37 commits into
mainfrom
SNOW-2364943

Conversation

@sfc-gh-yuwang

@sfc-gh-yuwang sfc-gh-yuwang commented Sep 29, 2025

Copy link
Copy Markdown
Collaborator
  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-2364943

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
  3. Please describe how your code solves the related issue.
    This PR aim to add feature of sending external telemetry information(trace/log) to an event table with open telemetry library.
    Here is the design doc of this feature:https://docs.google.com/document/d/1xUbIz5F-Bq3k5dt19SJN5dIy4QnvbQJ-Prq7pscEyAM/edit?usp=sharing

Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/event_table_telemetry.py
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated

@sfc-gh-vguttha sfc-gh-vguttha left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls address the doc comments. Rest LGTM. Thanks for doing this.

Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread tests/integ/test_external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/event_table_telemetry.py
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/event_table_telemetry.py
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/external_telemetry.py Outdated
Comment thread src/snowflake/snowpark/_internal/event_table_telemetry.py
Comment thread src/snowflake/snowpark/session.py Outdated
return self._sp_profiler

@property
def client_telemetry(self) -> ExternalTelemetry:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we rename to EventTableTelemetry to make it consistent?

Comment thread src/snowflake/snowpark/session.py Outdated
Comment thread tests/integ/test_external_telemetry.py Outdated
# test with mock exporter and authentication
with (
patch(
"snowflake.snowpark._internal.external_telemetry.create_attestation",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The patch path in the test doesn't match the actual module path. It should be snowflake.snowpark._internal.event_table_telemetry.create_attestation to match the import at line 10 of the event_table_telemetry.py file. This mismatch would cause the test to fail since the mock won't intercept the actual function call.

Suggested change
"snowflake.snowpark._internal.external_telemetry.create_attestation",
"snowflake.snowpark._internal.event_table_telemetry.create_attestation",

Spotted by Graphite Agent

Fix in Graphite


Is this helpful? React 👍 or 👎 to let us know.

ext.enable_event_table_telemetry_collection("snowflake.telemetry.events", logging.INFO, True)
tracer = trace.get_tracer("my_tracer")
with tracer.start_as_current_span("code_store") as span:
span.set_attribute("code.lineno", "21")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
span.set_attribute("code.lineno", "21")
span.set_attribute("code.lineno", "21")

logging.info("Trace being sent to event table")
ext.disable_event_table_telemetry_collection()
logging.info("out of scope log") # this log is not sent to event table
ext.enable_event_table_telemetry_collection("db.sc.external_et", logging.DEBUG, True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ext.enable_event_table_telemetry_collection("db.sc.external_et", logging.DEBUG, True)
ext.enable_event_table_telemetry_collection("snowflake.telemetry.events", logging.DEBUG, True)

logging.info("Trace being sent to event table")
ext.disable_event_table_telemetry_collection()
logging.info("out of scope log") # this log is not sent to event table
ext.enable_event_table_telemetry_collection("db.sc.external_et", logging.DEBUG, True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ext.enable_event_table_telemetry_collection("db.sc.external_et", logging.DEBUG, True)
ext.enable_event_table_telemetry_collection("snowflake.telemetry.events", logging.DEBUG, True)

Comment thread tests/integ/test_external_telemetry.py Outdated
@sfc-gh-yuwang sfc-gh-yuwang merged commit 975da52 into main Oct 24, 2025
26 of 28 checks passed
@sfc-gh-yuwang sfc-gh-yuwang deleted the SNOW-2364943 branch October 24, 2025 16:12
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 24, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants